Fix reviewer suggestion confidence for sole experts#4905
Merged
simonfaltum merged 1 commit intomainfrom Apr 6, 2026
Merged
Conversation
…coring The confidence calculation returned "low" when a single person was the sole contributor to all changed files, which is actually maximum confidence. Also, output files (out.*, output.txt) were scored 0.0 and completely excluded from reviewer analysis. They now get a small weight (0.01/total_files) so they contribute signal without dominating scores. Co-authored-by: Isaac
simonfaltum
requested review from
andrewnester,
anton-107,
denik,
pietern and
shreyas-goenka
as code owners
Eligible reviewersCould not determine reviewers from git history. Based on CODEOWNERS, these people or teams could review: @andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka Suggestions based on git history of 1 changed files (0 scored). See CODEOWNERS for path-specific ownership rules. |
denik
approved these changes
Apr 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The
suggest-reviewersGitHub Action had two issues:compute_confidencetreatedlen(ss) < 2as insufficient data, when it actually means there's one clear expert.out.*,output.txt) were scored0.0and completely excluded from analysis. Reviewers should still verify output changes make sense, so these files deserve a small (but non-zero) contribution to scoring.Changes
Before: Sole-author PRs always got "low" confidence. Output files were invisible to the scoring algorithm. PRs with exactly 2 contributors also always got "low" confidence (the
>= 3guards prevented any comparison).Now:
len(ss) == 1(sole contributor) returns "high" confidencelen(ss) == 2compares top vs second using the same 2x/1.5x thresholds0.01 / total_filesinstead of0.0, contributing signal without dominating scoresif weight == 0.0guard that could never trigger after the weight changeTest plan
compute_confidencebranches for correctnessThis pull request was AI-assisted by Isaac.